TITLE OF THE INVENTION 
COLOR REPRODUCTION SYSTEM 

BACKGROUND OF THE INVENTION 

The present invention relates to a color 
reproduction system capable of reproducing a color of 
an object in consideration of observation illumination 
light when the image recorded on a recording medium is 
to be reproduced* 

Recently, a color management system (CMS) such 
as a color reproduction system has been widely used, 
in which the image data obtained by photographing 
a desired object using a color image input device such 
as a digital camera is corrected, and a color of the 
object is faithfully reproduced by a display device 
such as a CRT monitor or an output device such as 
a color printer. 

In such a system, if illumination light on the 
photographing side on which an object is photographed 
differs from illumination light on the observation side 
on which a reproduced image is observed, the color 
based on the tristimulus values. X, Y, and Z of the 
object on the photographing side and observed under 
the illumination light on the observation side looks 
different from that under the illumination light on the 
photographing side owing to changes in the perception 
characteristics of a person, e.g., color adaptation. 
That is, a problem is posed in terms of "appearance". 
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The tristimulus values X, Y, and Z are the 
quantitative values of a color which are determined by 
the International Commission on Illumination WGIE and 
guarantee the "appearance" of the same color under the 
5 same illumination light. However, such values cannot 

properly cope with the "appearance" of a color under 
different kinds of illumination light as described 
above • 

1:3 In order to solve this problem, a conventional CMS 

10 aims at reproducing corresponding colors as tristimulus 

I J 

iQ values that provide the "appearance" of the same color 



on both the observation side and the photographing side 

by using a human - colors perception model such as a color 

A 

adaptation model. 



f;^ 15 Several models as human color perception models 

^.3^ including "Color adaptation models are disclosed in 

Mark. D. Fairchild, Color Appearance Models, Addison 
Wesley, (1998). Studies jba«. been made to construct 
models that allow more accurate color prediction. 
20 The conventional CMS reproduces the "appearance" 

of a color of an object on the photographing side. 
In contrast to this, the color reproduction system 
disclosed in Jpn. Pat. Appln. KOKAI Publication 
No. 9-172649 estimates a spectral reflectance image 
25 of an object from a photographing image, and applying 

an observation illumination light spectrum to the 
spectral reflectance image to obtain tristimulus values 



lit 
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under the observation illumination light and reproduce 
the color, thereby reproducing the "appearance" of the 
color on the observation side. 

In a method of performing such illumination 
5 conversion, since tristimulus values of a color of 

an object under actual observation illumination light 
are reproduced, the "appearance" of the color can be 
accurately reproduced without any consideration given 
'^3 to the human perception characteristics such as.- colog " 

10 adaptation. 
I;Q Unlike in the conventional CMS, in the above 

color reproduction method of performing illumination 
conversion, there is no need to establish a linear 
conversion relationship between the spectral 
15 sensitivity of a camera and CIEXYZ color matching 

functions, and a color adaptation model under study 
need not be used. This method, however, requires 
various data, e.g., illumination light spectrum 
data on the photographing side and observation 
20 side, the spectral sensitivity data of a camera, 

and statistical characteristics of the spectral 
reflectance of an object. 

In order to measure illumination light spectrum 
data of such data, an expensive spectrophotometer is 
25 generally required, and measurements under the actual 

photographing environment and observation environment 
are required. This makes it difficult to construct 



a simple system. 

In addition, the spectrum characteristics of 
observation illumination light are not always suited to 
reproduce a color of an object, a problem arises when 
delicate differences between colors are to be evaluated 

When there are a plurality of portions to be 
observed, these portions are rarely positioned under 
the same illumination light. In general, the 
respective observers use different kinds of illumina- 
tion light, and observe the color under the different 
kinds of illumination light. An inconvenience is 
caused in many cases, e.g., when an observer is to make 
a decision for diagnosis in a remote medical practice 
on the basis of colors while observing a color image 
of a patient and when absolute evaluation is required 
without any dependence on communication and observation 
environments . 

BRIEF SUMMARY OF THE INVENTION 
It is an object of the present invention to 
provide a color reproduction system which can replace 
measurement of an observation illumination light 
spectrum with simpler measurement of tristimulus values 
and performs illumination conversion that allows color 
reproduction of an object under an illumination light 
spectrum suited to color reproduction. 

According to the present invention, there is 
provided a color reproduction system comprising 



i:3 

I J 
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color image input means, color estimation means for 
calculating tristimulus values by correcting a color 
image signal obtained from the color image input means, 
and color image output means for outputting a color 
5 based on the tristimulus values obtained by the color 

estimation means, the color estimation means including 
illumination light measuring means for measuring 
tristimulus values of observation illumination light, 
virtual illumination light spectrum calculation means 
10 for calculating a virtual illumination light spectrum 

that provides tristimulus values equal to the 
tristimulus values of the observation illumination 
light which are obtained by the illumination light 
measuring means, and tristimulus value calculation 
15 means for calculating tristimulus values of the object 

under the virtual illumination light spectrum from the 
color image signal. 

The virtual illumination light spectrum 
calcul^tic^ means calculates a spectrum from a linear. 
20 ^- eu m of predetermined illumination light spectrum^ ^aaie ■ 
functions. The virtual illumination light spectrum 
Jl^^ (\ ^ calculation means calculates a spectrum ^ ^^rooponding 



O 



„J;<f* linear conversion • c e t.we en a product of a spectral 

sensitivity of the color image input means aijd 
25 a photographing illumination light spectrum ^nd ■ 



a product of a color matching function and the virtual 
illumination light spectrum. 
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In addition, there is provided a color 
reproduction system comprising color image input means, 
color estimation means for calculating tristimulus 
values by correcting a color image signal obtained by 
5 the color image input means, and color image output 

means for outputting a color based on the tristimulus 
values obtained by the color estimation means, the 
color estimation means including illumination light 
measuring means for measuring tristimulus values of 

10 observation illumination light, tristimulus value 

calculating means for calculating tristimulus values of 
the object under a predetermined standard illumination 
light spectrum from the color image signal, and 
corresponding color calculation means for calculating 

15 tristimulus values that provide "appearance" of the 

color of the object which is based on the tristimulus 
values under the standard illumination light spectrum. 

The color reproduction system having the above 
arrangement accurately reproduces the "appearance" of 

20 a color without being influenced by changes in the 

color perception characteristics of a person such as 
-eoiei* adaptation by reproducing the tristimulus values 
of an object under a virtual illumination light 
spectrum which provide tristimulus values equal to 

25 those of observation illumination light without 

measuring any observation illumination light spectrum. 
In addition, color reproduction can be performed 
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under illumination light superior in color rendering 
characteristics to an actual observation illumination 
light spectrum^ by calculating a spectrum from the 
^^^ ^^noa^LOum of baoic func t afena ^ — If sAoh ba-o-io f unct a 
5 are standardized, a spectrum is uniquely determined by 

the tristimulus values of observation illumination 
light. This facilitates standardization of the 
characteristics of observation illumination light 
on the spectrum level. Furthermore, if a virtual 

10 illumination light spectrum is obtained such that 

a linear conversion relationship is established between 
the product of the spectral sensitivity of the color 
image input means and a photographing illumination 
light spectrum and the product of a color matching 

15 function and the virtual illumination light spectrum, 

the accurate tristimulus values of an arbitrary object 
can be obtained under virtual illumination light. 

Each color of an object can always be observed 
under a predetermined standard illumination light 

2 0 spectrum without any dependence on photographing 

illumination light and observation illumination light 
by calculating the tristimulus values of the object 
under the predetermined standard illumination light 
spectrum and reproducing tristimulus values that 

25 provide the "appearance" of the color, based on the 

tristimulus values of the object under the standard 
illumination light spectrum, under the observation 
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tu 



5. J 



25 



illumination light , 

Additional objects and advantages of the invention 
will be set forth in the description which follows, and 
in part will be obvious from the description, or may 
be learned by practice of the invention. The objects 
and advantages of the invention may be realized and 
obtained by means of the instrumentalities and combina- 
tions particularly pointed out hereinafter. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 

The accompanying drawings, which are incorporated 
in and constitute a part of the specification, illust- 
rate presently preferred embodiments of the invention, 
and together with the general description given above 
and the detailed description of the preferred embodi- 
ments given below, serve to explain the principles of 
the invention . 

FIG. 1 is a block diagram showing an example of 
the arrangement of an image processing apparatus 
according to the first embodiment; 



FIG. 2 is a graph showing three ^asia functions 
for daylight spectra having wavelengths of 380 nm 
to 7 80 nm; 

FIG. 3 is a block diagram showing the detailed 
arrangement of a color correction device in FIG. 1; 

FIG. 4 is a block diagram showing the arrangement 
of a color correction device in an image processing 
apparatus according to the second embodiment; 
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FIG, 5 is a block diagram showing the arrangement 
of a color correction device in an image processing 
apparatus according to the third embodiment; 

FIG, 6 is a view showing an example of how an 
5 image processing apparatus according to the fourth 

embodiment is used; 

FIG, 7 is a view showing an example of the 
arrangement of a^ film * turret of a multispectral camera 
l;3 (MSG) used in the fourth embodiment; 

S.J 

10 FIG. 8 is a view showing an example of the 

i;g arrangement of the multispectral camera (MSG) used in 

i;n the fourth embodiment; 

FIG. 9 is a view showing an example of how an 
image processing apparatus according to the fifth 
■i^ 15 embodiment is used; and 

FIG. 10 is a block diagram showing the arrangement 
of an illumination light tristimulus value calculating/ 
processing section in the fifth embodiment. 

DETAILED DESCRIPTION OF THE INVENTION 
2 0 Embodiments of the present invention will be 

described in detail below with reference to the views 
of the accompanying drawing. 

An outline of a color reproduction system 
according to the present invention will be described 
25 first. 

In general, to reproduce a color of an object 
under observation illumination light from a spectral 



reflectance f ( A ) of the object, tristimulus values X^, 
Yq, and Zq must be reproduced, which are calculated 
from observation illumination light spectrum X ) and 
CIE color matching functions x( X ) , y( ^ ) , and z( X ) 
according to the following equations: 

780 

380 
780 

380 
780 

Zq = j z(X)E^(X)f(X)diX 
380 

In this case, since tristimulus values X, Y, and Z 
in a case wherein the object is located under observa- 
tion illumination light are reproduced, color 
reproduction can be accurately performed without any 
consideration of the visual characteristics of a person 
which change in accordance with the characteristics of 
observation illumination light. 

In contrast to this, the color based on the 
tristimulus value X, Y, and Z calculated under 
illumination light different from the actual 
observation illumination light is generally perceived 
as a color with an "appearance" different from the 
actual "appearance" under the observation illumination 
light because the visual characteristics, e.g., 
chromatic adaptation, of a person changes. 



with regard to the color perception of a person, 
the colors based on the same tristimulus values X, Y, 
and Z are perceived as the same color even if the 
colors have different spectra. For this reason, under 
illumination light with the same tristimulus values, 
the visual characteristics remain unchanged even with 
different spectra . 

That is, even if the tristimulus values X, Y, 
and Z of an object under virtual illumination light 
having the same tristimulus values X, Y, and Z as 
those of actual observation illumination light and 
a spectrum different from that thereof are reproduced, 
the "appearance" of the color under the illumination 
light can be accurately reproduced. 

On the basis of such an idea, the tristimulus 
values X, Y, and Z of observation illumination light 
are measured, and a virtual illumination light spectrum 
that provides the same tristimulus values X, Y, and Z 
are calculated to reproduce the tristimulus values X, Y, 
and Z of the object under the virtual illumination 
light. This makes it possible to perform accurate 
color reproduction without measuring the spectrum of 
the observation illumination light. 

A virtual illumination light spectrum that 
provides the same tristimulus values X, Y, and Z 
as those of observation illumination light can be 
calculated with a certain degree of freedom, but can 
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be uniquely determined by using three predetermined 
independent. baD - i r o functions. 

If the predetermined r ^Qoio functions are 
represented by p j^i 1 ) (i = 1, 2, 3), and the measured 
tristimulus values of observation illumination light 
are represented by X^q, ^wo^ ^wof ^ virtual 

illumination light spectrum Eyi{ 1 ) can be given by 



10 



15 



20 



Eh(^) = X CiPi(A.) 
i = l 

^780 780 



(2) 



C2 



780 



J K{X)pi{X)dX J x(>.)p2(>.)dA, J MX)p2(X)dk 
380 380 380 

780 780 780 

J y(>.)Pi(>.)d>. J y(X)p2(X)diX J y(A,)p3(>.)dX 
380 380 380 

780 780 780 

J z(X)pi(X)dLX j z(A,)p2(A,)dA, j z(X,)p3(X)dA, 
^380 380 380 



^wo 



'^wo 



(3) 



As the - bacio functions, for example, the daylight 



>a«ij&^ functions disclosed in Publication CIE No. 15,2 
(1986) can be used. 

functions, can be defined 
by determining weighting coefficients for the 



A daylight spectrum is given as 
the linear sum of three ^ > a^sic^ v 




respective 4)0310 functions . 



FIG. 2 shows the three . ba - sic functions of^ G unlQJght '^ 
having a wavelength of 3 80 nm to 7 80 nm. 

In this case, when the tristimulus values X, Y, 
and Z are determined, these three weighting 
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coefficients can be determined. Therefore, a daylight 
spectrum corresponding to the tristimulus values X, Y, 
and Z can be obtained. Tristimulus values X^/ Y^/ and 
Zh of the object under the virtual illumination light 
5 spectrum Eh{ X ) is given by 

780 

380 
780 

Yh = Jy(X)Eh(^)f(>^)dX -W) 
380 

780 
380 

Although these tristimulus values differ from the 
10 tristimulus values Xq, Yq, and Zq of the object under 

the actual observation illumination light, the 
"appearance" of the object is not influenced by changes 
in the visual characteristics of the person due to the 
difference in illumination light. 
15 In this manner, color reproduction free from the 

influences of changes in the visual characteristics of 
the person can be performed by reproducing the 
tristimulus values X, Y, and Z of the object under the 
virtual illumination light spectrum that provides the 
20 same tristimulus values X, Y, and Z as those of the 

actual observation illumination light without measuring 
any illumination light spectrum. 

AS described above, by using daylight •fea&ie- 
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functions and the like, color reproduction can be 
implemented under an illumination light spectrum having 
excellent characteristics for color evaluation 
regardless of the characteristics of an illumination 
5 light spectrum on the actual obser\^ation side. 

In addition, by standardizing^-b«:s« functions 
for a virtual illumination light spectrum, color 
reproduction under illumination light having the same 
□ tristimulus values X, Y, and Z always reproduces 

10 a color under illumination light having the same 

spectrum. This makes it possible to unify observation 
illumination conditions for reproduced colors in 
different observation environments . 

Photographing signals R, G, and B of an object 
15 with the spectral reflectance f(^) which is 

photographed under a photographing illumination light 
spectrum Em( X ) by an RGB camera with spectral 
sensitivities r{ X ) , q{ X ) , and b( /L ) , and tristimulus 
values XfQ, Yfo, and Zf^ of the object under the 
20 observation illumination light spectrum Eo( X ) can be 

given by 



1::*; 
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780 

R = jr(X)Eni(X)f(^)dA, 
380 

780 

G = |g(X)Eni(X)f(A,)dX 
380 

780 

B = J b(A,)Eni(A,)f(^)dA, 
380 

780 

Xfo = / x(X)Eo(A.)m)dX 
380 
780 

Yfo = Jy(A-)Eo(A.)f(X)dA, 
380 
780 

Zfo = Jz(X)Eo(A.)f(X)d\ 
380 



(5) 



(6) 
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In order to obtain the tristimulus values Xfof Yfof and 
ZfQ of an arbitrary object from the signals R, G, and B, 
the tristimulus values must be expressed as follows 
with respect to an arbitrary spectral reflectance f(^): 



^fo 
Yfo 



G 



Zfo J 



m2i rn22 '^23 
\^31 ^32 ^33 J 



(7) 



15 



That is, the tristimulus values must satisfy 

^x(A.)Eo(A.)^ 
Y(X,)Eo(^) = ni2i m22 n>23 
j^z(X.)Eo(X,) j \^3\ ">32 "»33. 



'^mii mi2 mi3)fr(X)Eni(A,)l 
g(>,)Em(X) 
b(X)Em(X) 



(8) 



In practice, the system rarely satisfies such a 
condition. However, this condition can be satisfied by 
replacing the observation illumination light spectrum 
Eo( 1 ) with the virtual illumination light spectrum 
Eh(X) that satisfies equation (8). 

At this time, the tristimulus values Xf^, YfQ, and 
ZfQ of an arbitrary object under the virtual 
illumination light can be accurately obtained from the 
signals R, G, and B. 

When a color under virtual illumination light 
different from actual observation illumination light is 
to be reproduced, the "appearance" of the reproduced 
color can be kept unchanged under the condition that 
the tristimulus values Xq, Yq, and Zq of the 
observation illumination light coincide with the 
tristimulus values Xj^, Yj^, and of the virtual 
illumination light. 

To obtain the virtual illumination light spectrum 
Eh( ^ ) that satisfies this condition, a spectrum that 
minimizes an error function Exyz value EL^-j-her 

given by the following equation is obtained by 
repetitive calculation . 

Exyz = [^o - ^h^^ + t^o -Yh]^ + [Zo -Zh]2 --(9) 



780 

^Luther = | [x(X)Eh(A,) - {miir(X) -h mi2g(^) mi3b(A.)}Ejn(^)]2ca 
380 

780 

+ J [Y(^)Eh(^) - {m2ir(>.) + m22g(>^) + m23ba)}Em(M]2dX 
380 

780 

+ J [za)EY^{X) - {m3ir(X) + m32g(>.) + m23h(X)}E^a)]2diX 
380 

• -(10) 

Luther ^ Lather ^ ^^^uther 
dmii dmi2 5mi3 

^ Luther ^ ^ Luther ^ ^ •••(11) 
9m2i 5m33 

If both the function Exyz "tihe value E^uther "0", 

the tristimulus values of an object having an arbitrary 
spectral reflectance under virtual illumination 
light are obtained from camera photographing signals. 
In this case, the same "appearance" as that under the 
virtual illumination light can be observed under the 
actual observation illumination light. 

Consider a case wherein virtual illumination light 
is provided as only one predetermined spectrum. 

In this case, if the virtual illumination light is 
standard illumination light, tristimulus values Xg, Yg, 
and Zg of the object under a standard illumination 
light spectrum Eq{ X ) is given by 



780 

380 
780 

Ys = jYa)Esa)fa)dX (12) 
380 

780 
380 

The "appearance" of the color based on the tristimulus 
values Xg, Yg, and Zg under the standard illumination 
light differs from the "appearance" of the color under 
observation illumination light having the tristimulus 
values X, Y, and Z which differ from those of the 
standard illumination light. 

The tristimulus values X, Y, and Z of an object 
whose "appearance" remains unchanged under illumination 
light having different tristimulus values X, Y, and Z 
are termed as corresponding colors. The "appearance" 
of the color of the object under the standard 
illumination light can be reproduced under the 
observation illumination light by obtaining and 
reproducing the corresponding colors of the object 
having the tristimulus values Xg, Yg, and Zg under the 
observation illumination light. 

As a method of obtaining corresponding colors, 
a method of using the human color perception model 
obtained by modeling a chromatic adaptation mechanism 
is used. 

As such chromatic adaptation models, several 
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models such as the Von Kries model that is the basis of 
other models are proposed. Corresponding colors Xq , Y^, 
and Zq based on the Von Kries model and corresponding 
to the tristimulus values Xg, Yg, and Zg of an object 
under the standard illumination light are given by the 
following equation, provided that the tristimulus 
values of the standard illumination light are 
represented by X^g, Y^g , and Z^g, and the tristimulus 
values of the observation illumination light are 
represented by X^^f ^wo' Z^q- 



= 



X 



0 0 



wo 



^wo 



0 0 



-wo / 



'WS 



0 0 



'■ws 



0 0 



M 



(13) 



-ws J 



Note that M represents a 3 x 3 matrix for converting 
15 the tristimulus values X, Y, and Z into the stimulus 

amount of a cone* In addition to such a chromatic 
adaptation model based on only the consideration of 
chromatic adaptation, many color "appearance" models 
for predicting perception amounts have been provided. 
2 0 These models can be used in accordance with different 

observation environments . 

In this manner, the tristimulus values of the 
object under the standard illumination light are 
obtained, and the "appearance" of the color based on 



the tristimulus values under the standard illumination 
light is reproduced under observation illumination 
light. With this operation, the "appearance" of a 
color of an object under the same illumination light 
5 spectrum can be reproduced under different observation 

illumination light environments. 

This makes it possible to standardize observation 
illumination light environments for an object 
independently of illumination light on the photograph- 
10 ing side and observation side. If a spectrum having 

excellent color rendering characteristics is defined 
as a standard illumination light spectrum used in 
'''''' this case, color reproduction can be performed with 

f7 an illumination light spectrum having excellent 

1;^ 15 characteristics independently of illumination light on 

the photographing side and observation side. In this 
case as well, the spectrum of observation illumination 
light need not be measured, and the tristimulus values 
X, y, and Z can be measured instead. 
2 0 FIG. 1 shows the arrangement of an image 

processing apparatus according to the first embodiment 
of the present invention. FIG. 3 shows the detailed 
arrangement of a color correction device 5 in FIG. 1 
and the flows of data ("[]" indicates the reference 
2 5 symbol of each data in FIG. 3). 

The image processing apparatus of this embodiment 
is designed as a virtual illumination color 
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reproduction system constituted by a photographing 
side A and observation side B connected to each other 
through an arbitrary line capable of transferring data 
such as image data. 

The photographing side A includes an RGB color 
camera 1 for photographing an object 3 under 
photographing illumination light 2, simplified 
spectrometer 4 for measuring a photographing 
illumination light spectrum, and the color correction 
device 5 for calculating the tristimulus data of 
the object 3 and converting it into an input signal 
to a CRT monitor 6 by using monitor profile data. 
The observation side B includes the CRT monitor 6 and 
a illumination light colorimeter 7 for measuring the 
tristimulus values of observation illumination light 8. 

On the photographing side A, the RGB color 
camera 1 photographs the object 3 under the 
photographing illumination light 2 to obtain an RGB 
image. The simplified spectrometer 4 then measures 
photographing illumination light spectrum data [MS]. 
For example, this simplified spectrometer 4 is designed 
to measure the radiation intensity of an illumination 
light spectrum in the wavelength range of 380 nm to 
7 80 nm in 1-nm intervals. Obtained RGB image data 
[CRGB] and photographing illumination light spectrum 
data are sent to the color correction device 5. 

On the observation side B, the illumination light 
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colorimeter 7 mounted on the CRT monitor 6 measures 
tristimulus values [IXYZ] of the observation 
illumination light 8. 

The illumination light colorimeter 7 is made up of 
5 three sensors respectively having spectral sensitivi- 

ties approximated to tristimulus value XYZ color 
matching functions by three filters (not shown), and 
measures the tristimulus values X, Y, and Z of the 
observation illumination light 8 near the CRT monitor 6, 
10 The tristimulus values X, Y, and Z of the 

observation illumination light 8 are sent to the 
color correction device 5 on the photographing side. 
The color correction device 5 calculates spectral 
reflectance data [f] of the object 3 from spectral 
15 sensitivity data [h] of the RGB color camera 1 and 

object characteristic data [ a ] which are stored in 
advance . 

Virtual illumination light spectrum data [OS] is 
calculated from the tristimulus value data of the 
2 0 observation illumination light 8 measured by the 
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urea oy ti 



illumination light colorimeter 7 and a tfea&ie- function 

A" 

(ji/ [P] stored in advance. Tristimulus value data [OXYZ] 

of the object 3 is calculated from the spectral 
reflectance data of the object 3, color matching 
2 5 function data [CMF] stored in advance, and the 

calculated virtual illumination light spectrum data. 
The obtained tristimulus value data of the 



object 3 is converted into an input signal [MRGB] to 
the CRT monitor 6 by using monitor profile data [MTP]. 
This input signal is then sent to the CRT monitor 6. 
The observer observes the color image of the object 3 
displayed on the CRT monitor 6 under the observation 
illumination light 8. 

The detailed arrangement of the color correction 
device 5 will be described next with reference to 
FIG, 3, 

The color correction device 5 is made up of 
a spectral reflectance calculator 9 for calculating the 
spectral reflectance data of an object from RGB image 
data, virtual illumination light spectrum calculator 10 
for calculating a virtual illumination light spectrum 
from the tristimulus value data of the observation 
illumination light 8, tristimulus value calculator 11 
for calculating the tristimulus values of the object 3 
from the spectral reflectance data of the object 3, 
output signal calculator 12 for converting the 
tristimulus values of the object 3 into an input signal 
to the CRT monitor 6, and storage device 13. 

In this arrangement, the spectral reflectance 
calculator 9 calculates a spectral reflectance f(^) of 
the object 3 from the RGB image data input from the RGB 
color camera 1 by using photographing illumination 
light spectrum data Si4( X ) measured by the simplified 
spectrometer 4, the object characteristic data stored 
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in the storage device 13, and the spectral sensitivity 

data of the RGB color camera 1, 

The photographing illumination light spectrum data 

Si^^( X ) has radiation intensity data at 1-nm intervals in 

5 the wavelength range of 380 nm to 780 nm. 

The object characteristic data is comprised of 

^ threey^mryic function data o \ {X ) , 02^^)9 ^Tid a 3 ( A ) 

of the spectral reflectance of the object 3. Each 

- ^k o^ Q ^^unction has values at 1-nm intervals in^ the 

rs.^l 10 wavelength range from 380 nm to 780 nm* Eachiba-sie- 

vn function . is *-#ei?ffl ed in accordanco with the typo-o - f -^ 

-efe^-ee-fe^ and an eigenvector of the correlation matrix of 

Dpcctral rof loGtanoo — ■ 

The spectral reflectance pf an object can be 

15 expressed by the linear ^ ouift of - ba s ic functions. 

A < 

The spectral sensitivity data of the RGB camera has 
sensitivity value data at 1-nm intervals in the 
wavelength range from 380 nm to 7 80 nm of ) , hg( A, ) , 

and hb( /L ) for each of R, G, and B channels. 
20 The spectral reflectance calculator 9 calculates 

the spectral reflectance f ( /L ) according to the 
following equations : 

f(X.) = a • (D-^C) • -(14) 
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a = (ai(X) ,02(X) ,a3(X) 
C = (R, G, B)t 

''780 780 780 

Jhr(X)ai(A.)Sm(A.)dX J (A,)a2(MSin(X)dA, J (^)a3(X)Snia)dA. 



D = 



380 



380 



380 



780 780 780 

Jhg(A.)ai(X)Snj(X)dA, J hg(X)a2(MSni(X)dX J hga)CT3(>,)Sm(A,)d^ 
380 380 380 

780 780 780 

Jhba)ai(X)Sni(X)dA, | hb(X)cy2(A.)Sin(^)clA, J hb(X)CT3a)Sm(X)dX 

V380 380 380 , 

•••(15) 



where • represents an inner product, t represents 

transposition, and -1 represents an inverse matrix. 

The virtual illumination light spectrum calculator 

10 receives tristimulus values X^q, Y^q, and Z^^q of 

10 observation illumination light measured by the 

illumination light colorimeter 7 and predetermined 

'^^ ^^^x^ ^xiTioX.xons pi(A), P2(^)/ p 3 ( A, ) of a 

A 

daylight spectrum from the storage device 13, and 
outputs a virtual illumination light spectrum Eh(A)« 

15 The virtual '^'^■'■^^'^^^^^^^^^^^^)^^ spectrum E^C ^ ) is 

the linear^&tt^-^^ff^^^^ functions pi(Jl), P o^^) t 

A /t 

and jo 3 ( A ) . The tristimulus values based on the 
virtual illumination light spectrum Eh(A) are the 
tristimulus values X^q, Y^q, and Z^q of the observation 
20 illumination light 8 and calculated according to \ 

equations (2) and (3). 

The tristimulus value calculator 11 calculates 



the tristimulus values X, Y, and Z of the object 3 
under the virtual illumination light spectrum from 
the spectral reflectance f{X) of the object 3 
calculated by the spectral reflectance calculator 9, 
color matching function data x( A ) , Y{ ^ ) r and z{ X ) 
stored in the storage device 13 in advance, and the 
virtual illumination light spectrum Eyii ^ ) calculated by 
the virtual illumination light spectrum calculator 10. 

Tristimulus values X^, Yj^, and of the object 
under a virtual illumination light spectrum are 
calculated according to equations (4). 

The output signal calculator 12 converts the 
tristimulus values X^/ Y^, and Z^ of the object 3 into 
input signals Rj^, Gj^r and Bj^ for display on the CRT 
monitor 6 by using monitor profile data. 

The monitor file data is made up of a 3 x 3 matrix 
and R, G, and B tone correction data. As this data, 
data obtained in advance by measurement is stored in 
the storage device 13. The tristimulus values X^f Y^/ 
and Zh are converted into values R^, Gl, and Bl having 
a linear relationship with the values X^f Y^/ and Z^ by 
matrix conversion. The values Rl, Gl, and B^ are 
converted into input signal values Rj^/ G^i, and Bj^ by 
the inverse functions of y ri^] f 'Vg[G], and Vbt^] that 
define the relationship between output luminances and R, 
G, and B inputs to the CRT monitor. The input signals 
Rj^, Gjn, and Bj^ are calculated from the tristimulus 



values Xh, Yh, and according to the following 
equation : 

Gm = Yi^[GL] •••(16) 







f \ 
rn^cx' ^yr* '^zr* 
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v^h; 



The input signals Rj^f G^^, and calculated by 
the color correction device 5 are output to the CRT 
monitor 6 • The CRT monitor 6 displays a color image 
having each pixel represented by the input signals Rj^, 
Gj^, and Bj^. The observer observes the image under the 
observation illumination light 8. In order to allow 
the observer to adapt to the observation illumination 
light 8 during observation, the CRT monitor 6 does not 
display "white" unique to the CRT monitor 6 

FIG. 4 shows the arrangement of a color correction 
device for an image processing apparatus according 
to the second embodiment of the present invention. 
Since the arrangement of this embodiment is the same as 
that of the first embodiment described with reference 
to FIG, 1 except for the color correction device, the 
same reference numerals as in FIG. 1 denote the same 
parts in FIG. 4, This color correction device will be 
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described in detail below. "[]" indicates the 
reference symbol of each data shown in FIG. 4. 

A color correction device 14 of this embodiment is 
comprised of a virtual illumination light spectrum 
5 calculator 10 for forming a tristimulus value XYZ 

conversion matrix under virtual illumination light 
from the R, G, and B values of a camera photographing 
signal, tristimulus value calculator 11 for calculating 
□ the tristimulus values of an object under virtual 

10 illumination light, output signal calculator 12 for 

l.iJ 

i;y converting the tristimulus values of an object 3 into 

e'l s 

m an input signal [MRGB] to a CRT monitor 6, and storage 

I device 13 . 

Each component of the above color correction 
15 device will be described in detail below. 

The virtual illumination light spectrum calculator 
10 generates a virtual illumination light spectrum from 
photographing illumination light spectrum data [MS] 
measured by a simplified spectrometer 4, spectral 
20 sensitivity data [h] of an RGB color camera 1 which is 

stored in the storage device 13 in advance, color 
matching function data [CMF], and tristimulus values 
[IXYZ] of observation illumination light which are 
obtained by an illumination light colorimeter 7, and 
25 also forms a conversion matrix [MTX] for calculating 

tristimulus values Xj^/ Y^, and [OXYZ] of the object 

\ 

under virtual illumination light. 
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Note that the virtual illumination light spectrum 
is obtained such that the values given by equations (9) 
and (10) are minimized or become equal to or less than 
a threshold. Such a spectrum is obtained by adjusting 
the spectrum value of each wavelength and repeatedly 
calculating the evaluation values of equations (9) 
and (lO). A matrix having values m^ to m33 calculated 
by equation (11) as elements in the virtual illumina- 
tion light spectrum determined in this manner is output 
to the tristimulus value calculator 11. 

The tristimulus value calculator 11 converts the 
RGB image data obtained by the RGB color camera 1 into 
tristimulus values X, Y, and Z using the conversion 
matrix obtained by the virtual illumination light 
spectrum calculator 10. 

The output signal calculator 12 is equivalent to 
the output signal calculator 12 shown in FIG. 2, and 
a description thereof will be omitted. 

FIG. 5 shows the arrangement of a color correction 
device for an image processing apparatus according to 
the third embodiment of the present invention. Since 
the arrangement of this embodiment is the same as that 
of the first embodiment described with reference to 
FIG. 2 except for the color correction device, the same 
reference numerals as in FIG. 2 denote the same parts 
in FIG. 5. This color correction device will be 
described in detail below. "[]" indicates the 
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reference symbol of each data shown in FIG. 5. 

A color correction device 15 is comprised of 
a spectral reflectance calculator 9 for calculating 
spectral reflectance data [f] of an object 3 
5 (not shown) from RGB image data, tristimulus value 

calculator 16 for calculating tristimulus values [SXYZ] 
of the object 3 under standard illumination light 
from the spectral reflectance data of the object 3, 

O corresponding color calculator 17 for calculating 

a, 

10 tristimulus values [CXYZ] of a corresponding color of 

I J 

(;a the object under observation illumination light from 

in the tristimulus values under the standard illumination 

!i ' ' 

~: =, 
: : : 

light, output signal calculator 12 for converting the 
I'T tristimulus values of the corresponding color into 

J^^ 15 an input signal [MRGB] to a CRT monitor 6, and storage 

'■'^ device 13, 

The tristimulus value calculator 16 and 
corresponding color calculator 17 in the color 
correction device 15 which are different from those 
20 in FIG. 3 will be described below. 

The tristimulus value calculator 16 calculates 
tristimulus values Xg/ Yg, and Zg [SXYZ] of the object 
3 under standard illumination light on the basis of 
spectral reflectance data f(^) [f] of the object 3 
25 calculated by the spectral reflectance calculator 9, 

together with color matching function data x(/L), y(^)/ 
and z(X) [CMF] and standard illumination light spectrum 



I : i 
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data Es(^) [SS] which are stored in advance. CIE D65 
spectrum distribution data for the standard 
illumination light spectrum Es(A) are given at 1-nm 
intervals in the wavelength range from 380 nm to 780 nm, 
5 The CIE D65 spectrum distribution is defined by 

relative values. In this case, however, this data is 
defined by absolute value data suited to the observa- 
tion environment for the standard CRT monitor 6. 

The corresponding color calculator 17 converts the 

10 tristimulus values Xg, Yg, and Zg of the object 3 under 

standard illumination light which are calculated by the 
tristimulus value calculator 16 into a corresponding 
color under observation illumination light. 

Tristimulus values X^g, Y^g , and Z^g [JXYZ] of 

15 standard illumination light are input from the storage 

device 13, and tristimulus values X^q, Y^q' ^wo 
[IXYZ] of observation illumination light are input from 
an illumination light colorimeter 7. 

Tristimulus values X^ ^ Y^, and Z^ of the 

2 0 corresponding color are calculated according to 

equation (13). The data of a matrix M represented by 
equation (13) is stored in the corresponding color 
calculator 17, and the following value is stored. 



25 M = 



^ 0.071 0.94 -0.016"^ 



-0,461 1.360 0.101 



^ 0,000 0.000 1.00 y 



(18) 
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The tristimulus values x^, Y^;,, and [CXYZ] of 
the corresponding color are converted into monitor 
input signals Rj^, G^q, and B^^ [MRGB] by using a monitor 
profile* The color image having each pixel made of 
5 Rj^f Gj^, and Bj^ is displayed on the CRT monitor 6. 

The observer then observes the color image displayed 
on the CRT monitor 6 under observation illumination 
light. 

□ The observer can therefore observe the 

10 "appearance" of the color of the object, which is 

i:o obtained under the standard illumination light, under 

ji^ the observation illumination light. 

" FIG. 6 shows an example of how an image processing 

L according to the fourth embodiment of the present 

15 invention is used. 

In this embodiment, for example, a patient 2 3 as 
an object is photographed by a multispectral camera 
(MSC) 24 in a clinic 21 to obtain a still image. 

The MSC 24 has a filter turret 26, as shown in 
20 FIG. 8. Seven interference filters 25 having different 

transmission wavelength bands are loaded in the filter 
turret 26, as shown in FIG. 7. The MSC 24 receives 
an optical image having passed through a photographing 
lens 41 as 7-channel image data by using a CCD 42 
25 in synchronism with the filter turret 26 rotated by 

a motor 40. 

The image data of the patient 2 3 photographed 



m. 




by the MSG 24 is loaded as 7-channel still image 
data into a computer 27. A white background plate 2 8 
whose spectral reflectance can be regarded as a 
uniform value isplaced behind the patient 23. 



camera 24 measures the reflected light spectrum of 
photographing illumination light from the background 
plate 28. 

The measurement data is loaded into the computer 
27 and divided by the spectral reflectance of the 
background plate 2 8 which is stored in advance, thereby 
calculating a photographing illumination light spectrum 
The clinic 21 is connected to a hospital 31 through 
a communication line 32. The tristimulus values X, Y, 
and Z of the observation illumination light 35 measured 
by an observation illumination light measuring device 
3 4 placed near a CRT monitor 33 in the hospital 31 are 
sent to the clinic 21 through the communication line 32 

Assume that a virtual illumination light spectrum 
Eh( ^ ) is generated by the same method as that in the 
first embodiment on the basis of tristimulus values X, 
y, and Z of observation illumination light 35. 

After the virtual illumination light spectrum is 
calculated, the XYZ image data of the object 23 is 
calculated on the basis of virtual illumination light 
spectrum data, photographing data, the spectral 
sensitivity data of the multispectral camera 24 which 



A 




29 juxtaposed with the multispectral 
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10 



is stored in the computer 27 in advance, and object 
characteristic data* The XYZ image data is then sent 
to the computer 3 6 in the hospital 31 through the 
communication line 32. The tristimulus values X, Y, 
and Z are calculated according to equations (19) 
and (20). 



v2y 



= AB -1 



92 
93 

.91 J 



(19) 



Elements a^j and b^j of matrices A and B are given by 
780 780 

aij = J jEha)xi(X)<fa)f(A,'))Eina')hj(X')dXdV 
X, = 380X,'=380 



780 780 

j J Em(^)hi(X)(f(X)f(A.' ))En,(X' )hj(V )dkdV 

X = 380 A,'=380 



(20) 



In this case, "<>" indicates expected value calculation, 
and object characteristics <f(A,)f(l')> represents the 
expected value of the spectral reflectance of the skin, 

15 which is important for diagnosis and stored in advance 

in a storage device as a database. The CRT monitor 33 
in the hospital 31 converts the XYZ image data of 
the patient 23, which is sent from the clinic 21, into 
an RGB input signal to the CRT monitor 33 by using the 

20 profile data of the CRT monitor 33 which is stored in 

advance . 
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The RGB input signal image of the patient 23 
is displayed on the CRT monitor 33, An observer 37 
such as a doctor observes the color image on the CRT 
monitor 33 under the observation illumination light 35, 
5 A white background plate 38 is placed behind the 

CRT monitor 33 in the hospital 31, and consideration 
is given to the observer 37 to allow him/her to adapt 
to reflected light of the observation illumination 
light 35 from the background plate 38. 

10 If a spectral reflectance p( /L ) of the background 

plate 38 is not a uniform value, tristimulus values Xq, 
Yq, and Zq of the observation illumination light 35, 
which are used to calculate a virtual illumination 
light spectrum, are replaced with tristimulus values X^/ 

15 Yb, and of the background plate 38 which are given 

by 

780 

Xb = j Ka)E^{X)p(X)dX 

380 
780 

Yb = jya)Eo(>.)p(>.)d>. • -(21) 

380 

780 

2b = j z(X)Eo(X)p(X)diX 
380 

2 0 The observation illumination light measuring 

device 34 directly measures the tristimulus values of 
the observation illumination light 35 or reflected 
light of the observation illumination light 35 from the 
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background plate 38 which are given by equations (21). 

Assume that the observation illumination light 
measuring device 34 also serves as a measuring 
device for generating the monitor profile of the CRT 
5 monitor 33. 

FIG. 9 shows an image processing apparatus 
according to the fifth embodiment of the present 
invention. This embodiment of the present invention 
has the following arrangement. In the fourth 

10 embodiment described above, the observation 

illumination light measuring device is used as 
a device for measuring the tristimulus values of 
observation illumination light. 

To accurately measure the tristimulus values 

15 of observation illumination light, the spectral 

sensitivity of the observation illumination light 
measuring device must have a linear conversion 
relationship with a color matching function. 

As a substitute for a measuring device having 

20 such special sensitivity characteristics, a combination 

of a color chart 39 including nine color chips whose 
spectral reflectances are -knovm- and a H cn o w n digital 
camera 4^^^^ used. The color chart 39 is placed 
near the CRT monitor, and the reflected light of 

25 observation illumination light reflected by each 

color chip is photographed by the digital camera 40. 
The photographing signal based on each color chip 



is sent to a computer 36 in a hospital 31 through 
a conununication line 41. An illumination light 
tristimulus value calculating/processing section of 
the computer 36 in the hospital 31 calculates the 
tristimulus values of observation illumination light 
from the photographing signal based on each color chip, 
the spectral sensitivity data of the digital camera 40 
which is stored in advance, the spectral reflectance 
data of each color chip, and color matching function 
data, and sends the tristimulus values to a computer in 
a clinic (not shown) through a communication line 42. 
FIG. 10 shows the arrangement of the illumination light 
tristimulus value calculating/processing section. 

A conversion matrix calculator 51 calculates 
a conversion matrix MTX for converting photographing 
signals R, G, and B of the respective color chips, 
which are sent from a color chip photographing signal 
input section 55, into tristimulus values X, Y, and Z 
of observation illumination light. The conversion 
matrix calculator 51 calculates this matrix from 
spectral sensitivity data h of the digital camera 40 
which is stored in advance in a digital camera spectral 
sensitivity data storage section 54 placed in the 
computer 36, spectral reflectance data CSR of each 
color chip stored in a color chip spectral reflectance 
data storage section 53 in advance, and color matching 
function data CMF stored in a color matching function 
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data storage section 52 in advance. 

The above conversion matrix MTX is calculated by 



M = 



3e. 



Cyl Cy2 . . •Cy27 
C2I C22 • • •Cz27 j 



ac 



yk 



dC 



zk 



Ei = 



780 

/ 



(k = 1-27) 



27 



xyzi(A,)- XCikSk(X) 
k = l 



dA, (i = X, y, z) 



Sk(M = ki(>.)fj(>.) (k = k(i, j) ) 

where xyzi(A) (i = y, z) is a color matching 
function, hi( 1 ) ( i = 1 to 3) is the spectral 
sensitivity of the digital camera 40, and fj{X) (j = 1 
to 9 ) is the spectral reflectance of each color chip. 

The conversion matrix MTX calculated by 
the conversion matrix calculator 51 is sent to 
a tristimulus value calculator 56. The tristimulus 
value calculator 56 calculates the average signal 
value of each color chip from the color chip images 
input from the color chip photographing signal input 
section 55, and calculates the tristimulus values X, Y, 
and 2 of observation illumination light by multiplying 
each average signal value by the conversion matrix MTX. 
The tristimulus value calculator 56 then stores the 
tristimulus values X, Y, and Z in a tristimulus value 
storage section 57. 
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In this embodiment, nine color chips are used. 
However, the number of color chips is not limited 
to this. In addition, a digital camera having 
sensitivities corresponding to three channels may be 
5 used as a multispectral camera; or digital camera 

having four channels or more, as on the photographing 
side . 

If the tristimulus values of observation 
l;3 illumination light are measured by using a digital 

= 10 camera and color chips in this manner, any expensive 

i J 

measuring device need not be used. In addition, since 
;=2 the spectral sensitivity characteristics of a general 

digital camera generally has no linear conversion 
relationship with a color matching function, 
1'=^ 15 tristimulus values cannot be accurately measured. 

k;j However, with a combination of a digital camera and 

color chips, tristimulus values can be measured with 
higher precision. 

As has been described in detail above, according 
2 0 to the present invention, there is provided a color 

reproduction system which can replace measurement of 
an observation illumination light spectrum with simpler 
measurement of tristimulus values, and can perform 
illumination conversion that can reproduce a color of 
25 an object under an illumination light spectrum suited 

to color reproduction. 

Additional advantages and modifications will 
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readily occur to those skilled in the art. Therefore, 
the invention in its broader aspects is not limited to 
the specific details and representative embodiments 
shown and described herein. Accordingly, various 
modifications may be made without departing from the 
spirit or scope of the general inventive concept as 
defined by the appended claims and their equivalents* 



